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1, INTRODUCTION 

Recently, study on recognition of human gait using features extracted from Kinect sensor 
was extensily explored and investigated. This is because the usage of Kinect in data acquisition could 
simplifies process of extracting gait features since Kinect sensor automatically tracks walking human by 
generating all coordinate of 20 skeleton joints in the 3D space [1-3]. Furthermore, gait features that are 
generated from the Kinect was proven significant for human recognition purpose as reported by [4-6]. 
Parameter such as attire, lighting along with walking view are commonly investigated in order to develop 
robust recognition technique. 

Initially, recognition of human gait using Kinect has been explored specifically in lateral view. 
For instance, nine subjects were required to walk from right to left, in front of the Kinect for eight times 
[7, 8]. Then, 11 sets of static features consisting of height, length of legs, torso, both lower legs, both thighs, 
both upper arms and both forearms, and two sets of dynamic features; the length of step and the speed were 
extracted. The researchers further examined the potential gait features related to human recognition using 
Naive Bayes, One Rule (1R) and C4.5 algorithm. The outcome was a high recognition accuracy of 91% using 
combination of four static gait features and Naive Bayes as classifier. The four static gait features identified 
as significant were height, length of legs, length of torso and length of the left upper arm. Conversely, 
recognition of human gait in lateral view was further investigated in the study of [9-11]. In these studies, 
combination of dynamic and static features was found to be highly significant for human recognition as 
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compared to static and dynamic features solely. Additionally, recognition of human gait was further extended 
to other gait views such as frontal, oblique and multi-view. Similar to lateral view, human gait could be 
highly recognize using fusion of static and dynamic features with classifiers such as Multi-Layer Perceptron 
(MLP), Support Vector Machine (SVM) [12-15] and k-Nearest Neighbor (KNN) [16], for both frontal view 
and multi-view [17]. Meanwhile, for oblique view, to the best of our knowledge, only one study reported on 
the recognition of human gait in oblique view. The researchers performed recognition of 10 subjects walking 
in lateral, oblique and frontal views in front of the Kinect using WEKA application along with LibSVM and 
One Class Classifier were employed to classify gait pattern according to its group. The results attained 
showed that the most significant feature was the fusion of seven static features; the length of the right and left 
shoulder, height, the length of the right arm and the length of the right knee and right foot and the length of 
the right hip. Conversely, orthogonal least square (OLS) has been used to select the most significant features 
in pattern recognition area as reported in [18-20]. Primarily, OLS algorithm has been utilised in system 
identification [21, 22] and structure of classifier model [23, 24]. Though OLS has never been used in gait 
recognition for feature selection, based on past researches that reported good performance of OLS as 
discussed in [18, 19], the potential of OLS as feature selection will be investigated in this study. 
Therefore the recognition of human gait in oblique view along with OLS as features optimization and 
Support Vector Machine (SVM) as classifier will be further explored and investigated. 


2. RESEARCH METHOD 

Figure | depicts the recognition of human gait using OLS and SVM for oblique view. Firstly, video 
of walking human was recorded by the Kinect during data acquisition. The recorded video was captured at 30 
frames per second with resolution of 640 x 480. Then, skeleton joints generated from the Kinect are 
extracted. In order to identify skeleton joints related to human gait recognition, OLS is used to select the 
significant skeleton joints and further classify by SVM. To evaluate the significant skeleton joints, 
recognition accuracy is computed based on average of 10-cross validation method. 
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Figure 1. Recognition of human gait for oblique view 


During data acquisition stage, 30 subjects were required to walk obliquely, in front of the Kinect, 
as shown in Figure 2. Here, layout measurement employed was based on the standard measurement of 
Kinect. The red area indicates the covered monitoring area and area of walking by subjects is recorded. 
In order to gain normal walking patterns, the participants began walking outside of the covered area. 
The subjects were requested to walk repeatedly for 10 times and no restriction on clothing types. In addition, 
the participants are required to walk naturally using their comfortable walking style and speed. Next, skeleton 
joints within a full gait cycle are extracted in the feature extraction stage. This stage involves five steps; 
removal of empty video frames, normalization of skeleton joints, detection of gait cycle, synchronization of 
frame number and lastly, extraction of skeleton joints. In the first step, empty video frames recorded during 
data acquisition are removed. Then, skeleton joints are normalized at a constant size. From observation, 
an inconsistency size of human skeleton attained as the participant moves toward the Kinect. Here, the size of 
human skeleton at a half of the trimmed video was chosen as the fix size. In addition, relative movement of 
skeleton joints was computed by considering head joint as the reference point. After that, a full gait cycle was 
detected by tracing three consecutive local minima in the vector of distance between the joint of left ankle 
and right ankle [4]. The distance vector was computed using Euclidean distance (1), in z-axis of the joints, 
and then filtered and smoothed using Savitzky-Golay moving average algorithm as: 


d(a, b) = V (a, 7 b,)? (1) 
where a 1s the joint of left ankle and b is the joint right ankle. 
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Figure 2. Recording area with its layout measurement 


As consequences, one to two gait cycles with various frame numbers were attained for all walking 
sequences. The various frame numbers (12 to 26 frames) further can lead to intricacy in the recognition stage. 
Therefore, in the synchronization of frame number stage, spline interpolation was employed to standardize 
the frame numbers at a maximum frame number (26 frames). Lastly, in the feature extraction stage, skeleton 
joints in each xyz-axis; which also known as skeleton joint points, as listed in Table 1 were extracted within a 
full gait cycle, using (2). 


Feature vector = (jx, Jy Jz, * M) Xs (2) 
where j,, = number of skeleton joint in x-coordinate, j,,, = number of skeleton joint in y-coordinate, jz, = 


number of skeleton joint in z-coordinate, m = number of frame and s = number of sample. 


Table 1. Skeleton Joint Point with Its Label 


Number of joint Name of skeleton Number of aad Number of Name of skeleton joint 
So see Name of skeleton joint point ce ; 
point joint point joint point joint point point 
1 Hip_center_x 21 Hip_center_y 4] Hip_center_z 
2 Spine_x 22 Spine_y 42 Spine_z 
3 Shoulder center_x 23 Shoulder center_y 43 Shoulder center_z 
4 Head_x 24 Head_y 44 Head_z 
2) Shoulder left_x 25 Shoulder left_y 45 Shoulder left_z 
6 Elbow left_x 26 Elbow left_y 46 Elbow left_z 
7 Wrist left_x Zi Wrist left_y 47 Wrist left_z 
8 Hand left_x 28 Hand left_y 48 Hand left_z 
9 Shoulder right_x 29 Shoulder right_y 49 Shoulder right_z 
10 Elbow right_x 30 Elbow right_y 50 Elbow right_z 
1] Wrist right_x 31 Wrist right_y 51 Wrist right_z 
12 Hand right_x 32 Hand right_y 52 Hand right_z 
13 Hip left_x 33 Hip left_y 53 Hip left_z 
14 Knee left_x 34 Knee left_y 54 Knee left_z 
15 Ankle left_x 238) Ankle left_y 55 Ankle left_z 
16 Foot left_x 36 Foot left_y 56 Foot left_z 
iW Hip right_x = ei Hip right_y 57 Hip right_z 
18 Knee right_x 38 Knee right_y 58 Knee right_z 
19 Ankle right_x 39 Ankle right_y 59 Ankle right_z 
20 Foot right_x A0 Foot right_y 60 Foot right_z 


For standardization purpose, only gait features within one gait cycle for each walking sequence per 
participant was used for the next stage. In the feature selection stage, the OLS algorithm calculates Error 
Reduction Ratio (ERR) for each skeleton joint in the model. From these, the percentage reduction made by 
each skeleton joint with respect to the output Mean Squared Error (MSE) is derived. The significance of each 
skeleton joint is ranked according to its contribution in reducing error in the model. The higher the ERR 
value, the more significant the skeleton joint point is. In this study, the OLS is computed for m frames of 
each skeleton joint point. Hence, the input features for OLS computation was arranged for each frame as 
shown in (3): 
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Sir =SxX Jen ¥nJ Zn (3) 


where j,, = number of skeleton joint in x-coordinate, jy, = number of skeleton joint in y-coordinate, j,, = 
number of skeleton joint in z-coordinate, s = number of sample. 
In the OLS computation, the expected output y, was computed as follows: 


Ve = des fir 9: (4) 


where, m is the number of frame, fj, is the input features; 6; is the solution to the linear least squares 
problem for i = 1,...,m. With the use of Householder-based QR decomposition, the input features, fj, is 
transformed into an auxiliary model as shown in (5). 


Vie = in Wik Gi (5) 


where, W;, 1s orthogonal to one another and g; are constant coefficients. 
The estimates of the coefficients g; are given by: 


= 5 
Gi Dee Wik ( ) 


and the error expression for ERR, err; computed as: 


—~2 yN 2 

Gi Xe=1W "i 

CoN a (6) 
k=1” k 


then, average err, ERR is computed as: 


err; 


ERR = = (7) 

The skeleton joint point with the highest ERR was arranged at the top and the lowest ERR was 
placed at the bottom. Further, the input feature was arranged using (2) with numbers of arranged skeleton 
joint points, starting from 2 to 60 skeleton joint points with incremental of 2. 

As aresult, various numbers of feature set were obtained. In the recognition stage, SVM with linear, 
polynomial and Radial Basis Function (RBF) kernel were employed as the classifiers. Upon several 
experimental analysis, regularization parameter (C) for linear kernel was significant to vary at 0.001, 0.01, 
0.1, 1 and 10, for polynomial and RBF kernel at 0.00001 to 0.01 with | decade increments and 10 to 100 with 
10 increments, respectively. In addition, polynomial order (d) of polynomial kernel was experimented at 2 
and 3 and sigma (o) for RBF kernel varied from 10 to 200. For the purpose of the evaluation of recognition 
performance and parameter selection of the classifier model [22], the 10-fold cross validation [25, 26] 
was used. 


3. RESULTS AND ANALYSIS 

In this section, the results attained based on the proposed methodology will be elaborated and 
discussed. As tabulated in Table 2, it can be observed that the rank of skeleton joint points after the 
implementation of OLS. Obviously, the joint of hip center in x-axis dominates the first rank as it attained the 
highest ERR. Furthermore, it can be seen that most of skeleton joints in x-axis placed at the first 30 rank, 
followed by skeleton joints in y-axis and z-axis, respectively. 

Figure 3 shows the recognition accuracy attained at the optimal SVM model. High recognition 
accuracy attained at 48 skeleton joint points for SVM with linear kernel (90.67%), 44 skeleton joint points 
for SVM with RBF kernel (88.33%) and 38 skeleton joint points for SVM with polynomial kernel (86.33%). 
In order to verify the effectiveness of OLS as feature selection, recognition accuracy for features without 
OLS was computed too. 

Figure 4 depicts the comparison of recognition accuracy between the original features and OLS 
features. As can be observed, recognition accuracy are enhanced using features selected by OLS, for all SVM 
kernels category. 
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Figure 3. Recognition accuracy for various numbers of 
skeleton joint points accordance to OLS rank 


Figure 4. Recognition accuracy attained using 
original features and features selected by OLS 
for SVM 


Table 2. Rank of Skeleton Joint Points after OLS Implementation 


Joint label Joint name ERR Joint label Joint name ERR 

1 Hip_center_x 0.699097276 55 Ankle left_z 0.000636189 
21 Hip_center_y 0.085875078 26 Elbow left_y 0.000621815 
4 Head_x 0.042695 157 31 Wrist right_y 0.000621299 
1 Shoulder center_x 0.019806264 o7 Hip right_z 0.000615943 
5 Shoulder left_x 0.015747802 29 Shoulder right_y 0.000592885 
10 Elbow right_x 0.014847255 38 Knee right_y 0.00058833 1 
23 Shoulder center_y 0.013521678 43 Shoulder center_z 0.000537843 
9 Shoulder right_x 0.004801643 60 Foot right_z 0.00052 1329 
17 Hip right_x 0.003904192 48 Hand left_z 0.000487427 
13 Hip left_x 0.003405773 47 Wrist left_z 0.000480826 

8 Hand left_x 0.003336582 28 Hand left_y 0.000467799 
14 Knee left_x 0.003330543 37 Hip right_y 0.000453269 
11 Wrist right_x 0.003307623 46 Elbow left_z 0.000438526 
7 Wrist left_x 0.003 139608 41 Hip_center_z 0.00042 1997 
24 Wrist left_y 0.002716615 45 Shoulder left_z 0.000415265 
2 Spine_x 0.002675057 a! Wrist right_z 0.000407073 
22 Spine_y 0.002644155 53 Hip left_z 0.00040253 1 
6 Elbow left_x 0.002040701 42 Spine_z 0.000383909 
34 Knee left_y 0.001630870 36 Foot left_y 0.000355970 
12 Hand right_x 0.001394476 50 Elbow right_z 0.000353017 
19 Ankle right_x 0.001333185 54 Knee left_z 0.000348224 
16 Foot left_x 0.001321263 49 Shoulder right_z 0.000346255 
15 Ankle left_x 0.001209764 58 Knee right_z 0.000319731 
35 Ankle left_y 0.000919835 18 Knee right_x 0.000296275 
20 Foot right_x 0.000909513 32 Hand right_y 0.000266994 
25 Shoulder left_y 0.00082 1467 59 Ankle right_z 0.000239201 
24 Head_y 0.000785205 44 Head_z 0.000220702 
40 Foot right_y 0.000736709 56 Foot left_z 0.000208776 
33 Hip left_y 0.000674316 39 Ankle right_y 0.000201806 
30 Elbow right_y 0.000654517 a2 Hand right_z 0.000198280 


4. CONCLUSION 

In conclusion, recognition of human gait in oblique view using features selected by the optimized 
OLS and SVM has been discussed in this study. The results showed that the combination of OLS and SVM 
with linear kernel contributed to the highest recognition rate namely 90.67% with 48 skeleton joint points as 
compared to the other two kernels. Future work includes investigating the human gait recognition using other 
feature selection method namely linear discriminant analysis and other classifiers specifically Naive Bayesian 
classifier and Deep Learning Neural Network. 
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